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SUMMARY 


Captain  John  Parrish  is  upset--his  twenty-first-century  space  shuttle  control  system  has 
twentieth-century  problems.  His  data  bases  are  easily  erased;  he  can’t  talk  directly  with  all  his 
data  links;  and  he  has  to  type  in  order  to  complete  his  mission.  His  anger  surges,  then  calms, 
as  he  encounters  Second  Lieutenant  Mark  Fillmore.  Fillmore  explains  that  his  satellite  control 
system  doesn’t  have  those  antiquated  difficulties  because  of  AIRT,  the  Automation  Impacts 
Research  Testbed.  When  Parrish’s  curiosity  heightens,  Fillmore  suggests  a  meeting  at  the  Space 
SPO,  the  System  Program  Office  that  uses  AIRT,  for  a  better  too*  at  this  unit  capability. 

Major  D.T.  Tooms  gives  a  detailed  explanation  of  AIRT  and  the  operability  concept  that 
allows  designers  to  predict  the  impacts  of  automation  in  terms  of  operator  skills  and  human 
performance  from  a  system  perspective.  The  payoff  of  operability  studies  is  that  the  system 
on  the  drawing  board  can  be  "seen"  and  tested  before  any  hardware  is  produced,  thus  affording 
designers  and  users  the  capability  to  adjust  the  design  to  suit  operational  needs.  Tooms  also 
reminisces  about  AIRT,  which  is  actually  a  twentieth-century  concept  presently  under  development 
by  the  Air  Force  Human  Resources  Laboratory.  He  wonders  why  it  didn’t  catch  on  faster, 
because  better  and  more  cost-effective  systems  could  have  been  built  in  the  interim. 
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PREFACE 


The  purpose  of  the  present  report  is  to  illustrate  the  concept  of  "operability"  and  why  it  is 
necessary  to  assess  operability  early  in  the  design  of  systems.  Determining  the  operability 
of  systems  is  more  encompassing  than  the  valid,  yet  specific,  sciences  focused  on 
determining  button  sizes  or  appropriate  computer  screen  colors.  Instead,  it  concerns  the 
"usability"  of  systems  in  their  intended  operational  environment,  with  emphasis  on  the 
operator’s  perspective  and  his/her  workload,  errors,  and  feedback. 

To  avoid  technical  formalism,  this  report  intentionally  describes  this  new  concept  of 
operability  in  an  entertaining,  but  informative  format.  Set  in  the  twenty-first  century,  the  report 
is  actually  a  story  relating  numerous  examples  of  good  and  bad  operability  designs.  The  story 
also  describes  futuristic  tools  and  methods  that  can  be  used  to  design  better  systems  from 
an  operability  point  of  view.  The  story,  in  closing,  ties  its  futuristic  view  into  research  presently 
being  conducted  at  the  Air  Force  Human  Resources  Laboratory  under  Work  Unit  3017-08-06, 
Methodology  for  Evaluation  of  Automation  Impacts  on  Tactical  Command  and  Control  (C2) 
Systems.  Much  of  the  research  has  been  accomplished  through  contract  with  Bolt  Beranek 
and  Newman  (BBN),  Incorporated,  under  Contract  Number  F33615-87-C-0007. 

I  extend  my  appreciation  to  Dr.  Richard  Pew,  Dr.  Kevin  Corker,  and  their  staff  at  Bolt 
Beranek  and  Newman,  Incorporated,  for  they  have  given  us  a  glimpse  to  the  future  of  system 
acquisition  in  their  development  of  a  prototype  called  the  Automation  Impacts  Research 
Testbed,  or  AIRT. 

I  also  acknowledge  my  senior  staff  officials,  Colonel  Harold  Jensen,  Colonel  Donald 
Tetmeyer,  Colonel  James  Clark,  Mr.  Bertram  Cream,  and  Dr.  Lawrence  Reed.  Without  their 
support  and  foresight,  it  would  not  have  been  possible  to  use  such  an  unusual,  but  informative 
format,  to  convey  such  a  significant  message. 


ii 


TABLE  OF  CONTENTS 


Page 


PROLOGUE  .  1 

ACTION .  1 

SUMMARY .  6 

EPILOGUE  .  7 


iii 


DESIGNING  HUMAN-CENTERED  SYSTEMS:  CIRCA  2039  SCENARIO 


PROLOGUE 

Space  Platform  Station  3.  Orbit  3B.  24  Nov  2039.  Captain  John  Parrish-disgruntled,  tired, 
and  frustrated-ambled  slowly  to  the  cafeteria  for  some  Turkey  Day  treats.  He  has  just  spent 
a  long  shift  on  the  Space  Shuttle  Mover  System  or  SPSMS.  Captain  Parrish  would  rather  call 
it  “SPASMS."  His  job  is  to  orchestrate  the  movement  of  space  shuttles  which  routinely  fly 
patterns  as  congested  as  Groundstation  Chicago-O'Hare  to  the  oldtimers.  Instead,  he  spends 
more  time  wrestling  the  system. 

"Thanksgiving,"  he  mumbled  to  anyone  who  would  listen.  "Boy,  would  I  be  thankful  for  a 
system  that  made  my  job  easier,  not  harder." 

“Sir,"  chirped  Second  Lieutenant  Mark  Fillmore,  who  is  wearing  the  distinctive  space  controller 
jump  suit,  "what  system  might  that  be?" 

"You  know,  ol’  SPASMS.  I’m  sure  you  have  just  as  many  problems  on  your  control  system 
as  I  do.  What’s  your  system?"  chided  Capt  Parrish. 

"Oh,  I  work  on  the  Satellite  Orbit  Control  System.  You  know,  the  new  one  we  call  ‘socks'! 
Why  don’t  you  like  SPASMS?"  Fillmore  inquired  as  he  followed  Parrish  to  a  table. 


ACTION 

As  they  sat  down,  Capt  Parrish  reluctantly  began  his  story. 

"It  all  started  this  morning  when  I  began  entering  my  database  into  the  system.  Jeez,  I’m 

no  typist.  It  took  almost  an  hour  to  enter  all  the  weather,  coordinate,  and  frequency  data. 
Heck,  by  then  I  almost  had  to  control  on  the  fly  because  shuttles  were  cornin’  at  me  left  and 
right!" 

"Sir,  in  SOCS  we  don’t  have  to  type  per  se.  We  have  direct  data  links  to  the  appropriate 
agency.  All  we  have  to  do  is  confirm  the  reception  with  one  membrane  panel  stroke  or  voice 
command,  our  choice-takes  about  3  minutes  total." 

"Oh  yeah?  Well,  I  bet  you  still  had  problems  when  they  hooked  up  that  new  StarBIDS. 

You  know,  the  space  version  of  the  old  Battle  Information  Display  System-BIDS.  We  had  to  tie 

it  into  an  existing  datalink  and  then  toggle  back  and  forth  when  we  needed  it.  Sure  the  link 
worked  from  an  engineering  standpoint,  but  talk  about  confusing!  Heck,  I  could’ve  been  talking 
to  Mars  for  all  I  knew.  They  knew  StarBIDS  was  coming,  but  the  system  was  already  set  in 
concrete.  I’d  really  like  to  put  it  in  concrete!" 

"Well,  Sir,  when  we  designed  SOCS,  we  anticipated  that  StarBIDS  was  coming.  In  fact,  we 
also  have  enough  capability  to  handle  GalBIDS  and  even  UniBIDS.  Flexibility  is  the  key  to 
keeping  up  with  technology." 

"Okay,  I  can  believe  that,  but  let  me  tell  you  this  one.  This  morning  I  took  a  short  break. 

I  returned  to  my  console,  sat  down,  put  on  my  helmet  display,  and  promptly  deleted  my 
database.  Another  hour  wasted-at  least  it  was  almost  lunchtime  when  there  are  fewer  shuttle 
flights." 
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"How  did  that  happen,  Sir?"  quizzed  the  lieutenant. 

"They,  the  engineers,  don’t  tell  you  everything.  If  you  take  your  helmet  off,  and  then  put 
it  back  on,  it'll  ask  you  the  question,  ‘Are  you  ready  to  start?'  You're  supposed  to  say,  ‘Continue,’ 
if  you’ve  taken  the  helmet  off  for  only  a  moment.  Instead,  I  said,  ‘Yes.’  My  database  was 
reinitialized." 

"Sir,  our  system  has  checks  and  balances  where  necessary.  We  made  sure  of  that.  In  fact, 
SOCS  would  have  asked,  ‘Are  you  sure?’  Those  things  just  don’t  happen  on  SOCS." 

“How  do  you  know  that?  Besides,  you’re  only  a  second  louie,"  remarked  Parrish. 

"A  fjisi  lieutenant  next  month,  Sir.  When  1  got  here  almost  2  years  ago,  I  got  in  on  the 
tail  end  of  helping  design  the  system.  Guys  like  us,  controller  types,  we  helped  de-program 
errors  and  ensure  flexibility." 

Captain  Parrish  offered  his  experience,  "Well,  I’m  an  old  hat,  prior  service;  made  technical 
sergeant.  We  sat  down  with  engineers,  too,  almost  10  years  ago.  Boy,  time  flies.  Anyway, 
we  told  ’em  what  we  needed.  Of  course  we  ail  transferred  before  it  was  built.  Never  knew 
I  would  have  to  come  back  and  work  on  this  piece  of.,  well,  whatever.  They  did  their  magical 
programming.  All  I  know  is  that  what  they  delivered  was  not  what  we  wanted  back  then,  and 
it  sure  doesn't  help  us  handle  all  the  shuttle  traffic  now.  The  engineers  put  all  their  marbles 
with  the  computer,  but  forgot  that  the  little  guy  has  to  use  it.  You  guys  just  got  lucky." 

Boldly,  the  lieutenant  disagreed,  "Beg  to  differ,  Sir.  We  didn’t  have  luck;  we  had  AIRT." 

"Art?  You  had  some  Leonardo  de  Vinci  on  the  design  team?"  quipped  the  captain. 

"No  Sir,  AIRT,  the  Automation  Impacts  Research  Testbed.  Actually  its  real  name  is  something 
like  the  Human  Factors  Operability  Workbench.  Its  predecessor  was  A...I...R...T.  We  prefer  to 
say  ‘Ail’ -a  lot  simpler." 

Somewhat  incredulous,  Parrish  asked,  "So  what  is  this  AIRT?" 

"Well,  let  me  explain,  Sir.  Suppose  you  could  sit  down  right  now  at  the  control  panel  of 
the  Hyperspace  Vehicle  Remote  Control  Panel...." 

"But  that’s  still  on  the  drawing  board-in  the  conceptual  phase,"  interrupted  Parrish. 

"Exactly,  Sir.  AIRT  gives  the  user  a  ‘mockup’  of  the  system  long  before  any  hardware  is 
manufactured.  You  see,  back  when  you  guys  helped  the  designers  and  engineers,  all  you  did 
was  sit  at  a  big  table  and  talk."  He  paused  and  thought,  "Dream  is  a  better  word."  Continuing, 
he  said,  "AIRT  gives  you,  the  user,  the  ability  to  ‘touch’  the  system  and  experiment  in  an 
operational  context.  Then  you  can  give  real  feedback  on  what  the  system  should  do  and  how 
it  sometimes  keeps  you  from  doing  what  you  have  to  do." 

Parrish,  obviously  becoming  impressed,  probed,  "Lieutenant,  that  sounds  like  it  might  work. 
Tell  me  more  ..." 

"Well,  Sir,  it  would  be  a  lot  better  if  we  transported  over  to  the  System  Program  Office. 
You  know,  Space  SPO  on  3A." 
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Space  Platform  Station  3.  Orbit  3A.  30  Nov  39.  "Hi,  I’m  Major  D.T.  Tooms.  I'm  glad 
Mark  called.  I  can  use  all  the  operational  opinions  I  can  get!"  exclaimed  Major  Donald  Thomas 
Tooms,  ever  the  salesman.  Major  Tooms  is  a  human  tactors/operability  expert  who  consults 
for  the  SPO  on  such  issues. 

As  the  three  men  walked  down  a  hallway  of  the  Space  SPO,  Parrish  noticed  that  there  were 
many  computers  and  supporting  hardware  in  the  adjoining  rooms.  He  wasn’t  familiar  with  all 
the  equipment,  but  he  did  recognize  the  helmet  displays,  plasma  panels,  liquid  crystal  displays, 
bio-electrical  interfaces,  and  the  ever  popular  electronic  "grease-pencil"  board.  In  particular, 
Parrish  was  puzzled  by  the  presence  of  several  operational  people;  these  were  his  buddies! 
"Funny,"  he  thought  aloud,  "all  I  remember  in  the  SPO  was  a  bunch  of  paper-pushers." 

Major  Tooms  responded,  "We  have  a  much  more  integrated  approach  now.  Most  of  the 
people  here  are  lab  technicians  working  with  subject-matter  experts;  your  colleagues  are  helping 
us  with  a  robotics-based,  extra-planetary  control  system.  However,  because  of  technology,  the 
operational  hands  are  also  designing  components  to  a  certain  degree.  Although  they  don’t  have 
any  formal  training,  the  operability  workbench  gives  them  the  ability  to  influence  the  design. 
But  we're  getting  a  little  ahead  of  ourselves  ...  Let’s  go  into  my  office." 

Maj  Tooms’  office  lacked  a  conspicuous  feature -a  traditional,  twentieth-century-style  desk. 
Instead,  Tooms  had  a  liquid  crystal  table.  With  a  laugh,  he  explained  that  his  desk  is  "simulated" 
on  the  left  half,  while  his  real  work  is  done  on  the  right  half. 

"The  right  half  of  the  LC  table,"  he  pointed  out,  "acts  as  an  integrated  pegboard,  if  you 
will.  All  the  systems  you  saw  as  you  were  walking  in  are  simulated  and  contained  in  my 
toolbox ."  He  deftly  touched  an  icon  resembling  a  toolbox  in  the  upper  right  of  the  LC  table. 
"The  box  opens,  almost  literally,  because  hologram  technology  displays  miniature  items  in  a 
3-dimensional  format." 

Tooms  continued,  "Now  each  of  these  icons  represents  a  piece  of  equipment.  We  call 
them  equipment  ‘objects.’  Objects  are  computer  code  that  contains  attributes.  Equipment 
objects,  which  represent  the  equipment  you  saw  outside,  consist  of  attributes  like  communication 
hook-ups,  functions,  capacity,  etc.  The  key  is  that  they  all  contain  similar  computer  code 
architectures;  therefore,  I  can  take  each  icon  and  quickly  connect  them  to  create  a  mockup 
of  a  system.  Watch." 

The  major  moved  some  of  the  objects  to  fashion  a  small  control  panel.  As  Tooms  stated, 
an  object  is  simply  a  cluster  of  computer  code  that  simulates  the  characteristics  of  a  particular 
piece  of  equipment  or,  in  other  cases,  a  complex  function  or  a  human  operator.  Each  object 
is  independent;  that  is,  objects  are  self-contained.  But  they  also  have  a  structure  that  allows 
them  to  easily  interact  with  other  objects.  This  feature  permits  Tooms  to  compose  a  small 
system  in  minutes. 

Immediately  after  Tooms  completed  the  mockup,  the  LC  table  dimmed,  but  highlighted 
compatibility  checks.  The  major  fine-tuned  some  of  attributes  of  the  equipment  objects,  and 
suddenly  the  table  brightened  as  if  to  approve  of  the  design-no  more  compatibility  checks. 

Tooms  offered  an  explanation,  "Now  I  have  just  created  a  quick  mockup.  But  don’t  confuse 
this  with  the  primitive  rapid-prototyping  of  the  last  century,  because  this  is  not  the  last  step  in 
the  process,  but  the  first.  Next  I  can  open  the  human  object  box."  It  looked  like  a  smiley  face. 

With  ease,  Tooms  placed  human  objects  with  respect  to  nodes  on  the  control  panel.  The 
table  dimmed  again,  but  correspondingly  sent  out  data  indicating  human  performance  parameters. 
The  output  also  indicated  that  there  was  an  operability  problem:  a  communication  bottleneck. 
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This  bottleneck  was  revealed  by  data  relating  to  the  human  operator’s  auditory  channel. 
Based  on  these  performance  data,  the  LC  table  displayed  seveial  solutions. 

Tooms  deciphered  the  readout,  and  boiled  it  down  to  two  alternatives:  reduce  the 
communication  or  modify  the  operator  requirement. 

Tooms  explained,  "Well,  we  can’t  reduce  the  communications;  so,  we’ll  have  to  change  the 
operators.  We  can  try  adding  an  operator  1  The  threesome  awaited  the  results. 

'Aha/'  Tooms  exclaimed,  "can’t  do  that.  Another  operability  problem. ..our  comm  stations 
can  t  support  an  additional  body  ..too  many  life  support  systems  required  and  no  space  to  put 
them  in.  Let’s  remove  the  additional  operator  and  add  a  communications  filter."  Again  they 
waited. 

Now,  we’re  headed  in  the  right  direction,"  said  Tooms.  "The  filter  fits,  but  we  still  have  to 
consider  what  communication  messages  are  really  required.  Time  to  go  see  the  user.  After 
gathering  data  from  the  user,  we  can  test  the  system  in  an  operational  context.  From  those 
tests  we  gather  more  human  performance  data  in  areas  such  as  visual,  cognitive,  psychomotor.. .as 

well  as  auditory  data." 


From  the  looks  of  this  system,"  Tooms  said  as  he  pointed  to  the  LC  table,  "we  would 
probably  key  on  the  auditory  data  since  we  identified  comm  as  a  problem  early  in  the  design. 
I  have  a  feeling,  though,  that  once  we  subject  the  human  operator  objects  to  a  scenario,  we’ll 
find  a  lot  of  interesting  cognitive  data  as  well.  Do  you  get  the  idea?  Now  keep  in  mind  this 
is  a  very  rudimentary  example,  but  you  can  see  that  by  interacting  with  the  potential  operators, 
and  by  applying  incremental  adjustments,  we  can  produce  a  rather  good  system.  Do  you 
realize  the  importance  of  this?" 

Before  Lieutenant  Fillmore  could  answer,  Capt  Parrish  blurted,  "Why  sure,  nothing  has  actually 
been  built!  You  can  run  all  kinds  of  tests  and  configurations  before  spending  a  bunch  of 
money  on  prototypes  and  production.  That’s  when  we  usually  find  out  that  it’s  difficult  to  use 
the  system  But  by  then  it’s  too  late  to  do  anything  but  retrofit  as  best  as  possible.  Sometimes 
the  retrofits  are  worse  than  the  original  problem  because  we  have  no  way  of  predicting  their 
impact." 

Exactly,"  agreed  Major  Tooms.  "Which  leads  me  to  my  next  point.  With  this  system, 
we  -actually  a  contractor  usually  does  what  I’m  showing  you-can  go  to  the  user  and  ask  how 
he  likes  it.  Normally  changes  can  be  roughed  out  in  a  short  time.  The  user  never  loses 
contart  with  the  contractor  during  the  design  process.  The  design  process  effectively  ends 
when  a  much  scrutinized  specification  is  produced.  Even  then,  with  modular  designs,  production 
changes  don’t  stop  the  presses  like  they  used  to  in  the  last  century.  All  along  the  user  has 
something  he  can  see  and  touch.  And  he  knows  that  what’s  going  to  be  built  is  consistent 
with  what  he  needs  because  of  the  interactive  process.  You  know,  when  I  see  a  system  built 
that  doesn’t  take  on  human  element  and  operability  issues,  I  just  wish  that  AIRT  had  come 
along  sooner.  Too  often  we  human-factored  the  buttons,  but  not  the  process  of  using  the 
buttons;  thus,  operability  was  ignored." 

Major  Tooms,  I  have  a  question,"  stated  Fillmore.  "What  exactly  is  this  operability  stuff?" 

"Well,  Mark,  it  addresses  a  whole  range  of  issues  that  simple  rapid-prototyping  can’t  deal 
with  alone.  For  instance,  many  older  contro'  systems  cannot  be  expanded.  Like  SPASMS,  the 
acquisition  community-by  which  I  mean  the  operators,  the  contractors,  the  Air  Force,  was  aware 
of  StarBIDS,  but  they  designed  the  system  to  spec  like  they  were  supposed  to  do  back  then. 
However  before  production,  they  didn't  fully  test  a  mockup  in  its  intended  operational  environment. 
Clearly  the  StarBIDS  issue  would  have  surfaced--and  in  time  for  a  solution.  As  it  turned  out, 
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SPASMS  was  built  to  spec;  in  fact,  it  made  starlines  because  it  was  under  cost,  on  schedule, 
and  worked.  Unfortunately,  it  was  not  designed  or  preliminarily  operated  in  a  context  consistent 
with  the  other  star  systems-existing  or  forthcoming;  so  dare  I  say  the  spec  was  not  altogether 
right?!" 

Capt  Parrish  inquired,  'Why  did  it  take  so  long  to  include  operability  in  systems?" 

Tooms  paused  to  think  over  his  words  and  began,  "The  whole  SPASMS  issue  is  what  really 
brought  about  the  emergence  of  the  operability  idea.  Actually,  operability  was  thought  about  a 
long  time  ago  in  a  laboratory  last  century-just  a  vision.  The  scientists  wanted  to  create  a 
technology  and  methodology  that  would  allow  preliminary  testing  of  systems  from  a  truly  systems 
point  of  view.  It  just  took  a  lot  of  wasted  money,  convincing,  and  a  system  like  SPASMS  to 
get  the  ball  rolling." 

Tooms  continued,  "Back  to  defining  operability.  I'm  not  specifically  talking  about  design 
issues  such  as  how  fast  the  computer  whirs,  nor  am  I  concerned  about  the  measurements  of 
a  switch  All  of  these  issues  are  valid  system  design  considerations,  but  operability  is  concerned 
with  the  system  in  use.  A  litany  of  tough  questions  must  be  asked-and  answered-up  front." 

Tooms  held  his  left  hand  up  and  extended  his  index  finger,  indicating  question  number  one: 
'What  is  the  impact  of  adding  automation?  Sure,  automation  provides  new  and  increased 
capability,  but  what  about  the  operator?  Can  he  use  it?  How  many  operators  are  needed  and 
what  will  they  do  using  the  new  capability?" 

The  next  finger,  is  an  operator  likely  to  ‘abuse’  the  system  by  using  it  as  a  high  tech 
version  of  the  old  system?  Better  yet,  can  we  even  train  the  operator  to  use  a  complicated 
system  to  its  fullest  capability?" 

On  to  the  ring  finger,  "Like  in  SPASMS's  case,  can  the  system  tie  into  other  systems?  What 
about  the  systems  coming  down  the  pike?  Technology  boomed  at  the  end  of  the  last  century; 
so,  the  military  rightfully  took  advantage  of  this  boom.  But  did  you  know  at  times  the  Pacific 
forces  could  barely  communicate  and  cooperate  with  the  Atlantic  forces  because  of  the  differences 
in  command,  control,  and  communication  systems?" 

Tooms  paused  and  then  held  up  his  little  finger,  "What  are  the  system  bottlenecks?  Indeed, 
a  computer  can  simultaneously  deliver  20  radio  messages,  but  can  an  operator  process  them, 
as  we  saw  in  our  little  mockup?" 

Finally,  with  increased  animation,  Tooms  held  up  his  thumb,  the  only  remaining  digit,  and 
said,  "Perhaps  the  most  serious  questions  to  answer  concern  mistakes.  Do  the  operators 
consistently  make  the  same  small  mistakes?  More  importantly,  do  they,  or  can  they,  make  big 
blunders  that  are  preventable?  In  most  older  systems,  we  didn’t  have  any  idea  what  mistakes 
could  be  made." 

By  now,  Tooms  was  throwing  both  arms  in  the  air,  "Let  me  give  you  an  example  from  the 
other  guys.  Remember  the  Chernobyl  nuclear  plant  disaster  in  the  1980’s?" 

Tooms,  seeing  their  acknowledgment,  resumed,  "It  was  a  system  error." 

Back  to  the  fingers,  Tooms  continued,  "Mistake  number  one:  human  error.  The  operators 
insisted  on  ignoring  warning  systems,  and  attempted  to  turn  off  some  danger  indicators." 

"Mistake  number  two.  equipment  error.  The  equipment  allowed  the  operators  to  turn  off 
warning  gauges.  Both  add  up  to  a  system  error,  i.e.,  operability. 
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Perspiration  seeping  from  his  brow,  Tooms  no  longer  hinted  at  his  sentiments  and  came 
right  out  with,  "You  know,  it  really  bugs  me  that  it  took  so  long  to  realize  the  potential  of  the 
operability  approach.  Let  me  relate  an  analogy.  From  time  to  time  in  your  career  you'll  present 
a  briefing  at  a  very  high  level,  maybe  to  a  few  stars.  Well,  I’ve  done  it  too  many  times  to 
count.  My  time  costs  money,  wouldn't  you  agree?  More  importantly,  generals  will  make  decisions, 
based  on  my  briefings,  that  impact  even  more  money.  Generals  also  present  briefings  to  higher 
levels,  such  as  Congress." 

"But  let’s  get  to  my  point.  We  all  do  dry-runs.  Why?  To  ensure  that  our  briefing  will 
provide  the  correct  message  to  an  inquisitive,  and  perhaps  hostile  audience.  We  ensure  that 
we  can  answer  the  tough  what-if  questions  during  dry-runs  to  prevent  our  future  embarrassment 
by  having  to  answer  them  unprepared.  Underlying  the  well-rehearsed  briefing  is  the  assurance 
that  our  program  stands  on  itc  own.  Why  has  it  taken  so  long  to  do  the  same  thing  for  our 
human/machine  programs?  We  need  to  dry-run  or  rehearse  them  before  it’s  too  late  in  the 
acquisition  process.  All  too  often  we’re  embarrassed  by  the  ‘operational  questions’  asked  of 
the  human/machine  system.  Systems  like  the  operability  workbench  give  us  the  ability  to  avoid 
the  operational  embarrassments." 

Almost  out  of  breath,  Tooms  rhetorically  asked,  ’And  what’s  the  real  payoff?  Well,  designing 
a  system  that  can  be  used  from  Day  1  after  fielding,  instead  of  fumbling  with  operational 
questions  after  it’s  built.  We’ve  demonstrated  that  we  can  build  great  control  panels,  exactly  to 
specs,  but  can  our  guys  effectively  use  them  in  the  real  universe?" 

"Furthermore  if  we’re  reeeally  good,"  Tooms  emphasized,  "we’ve  looked  far  enough  ahead 
so  that  new  equipment  doesn't  have  to  be  thrown  aside  whenever  a  new  threat  or  system 
appears.  Technology  does  not  win  wars,  it’s  the  total  system,  man  and  machine,  that  wins 
and  preserves  our  freedom.  That’s  what  operability  is  all  about,  Gentlemen.  Questions?" 

Parrish  almost  apologetically  asked,  "Major  Tooms,  what  about  those  ops  guys 
outside?" 

Tooms  explained,  "Sorry,  I  guess  I  got  on  my  soapbox  there.  The  guys  outside  are  helping 
us  tweak  the  systems  before  final  production.  At  this  point,  with  the  techniques  we  have 
available,  the  operators  can  mova  the  switches  a  bit  or  tune  the  frequencies,  kind  of  a  last-minute 
test  drive  before  the  assembly  line  is  cranked  up.” 

Parrish  reiterated,  "Ops  guys  designing  systems;  I  love  this  country!" 

With  no  other  questions,  both  visitors-drained  but  satisfied -stood  up,  expressed  their  gratitude 
and  started  to  exit.  Lieutenant  Fillmore  quickly  gained  a  lead  on  Captain  Parrish,  who  was 
still  eyeing  the  equipment.  Major  Tooms  noticed  his  curiosity.  Wanting  to  capitalize,  Tooms 
beckoned,  "Captain  Parrish,  want  to  make  a  difference  in  the  Air  Force?  C'mon  back.  I  could 
use  a  little  help.  ..” 

Ever  the  salesman. 


SUMMARY 

The  Air  Force  Human  Resources  Laboratory  is  presently  researching  and  developing  the 
Automation  Impacts  Research  Testbed,  or  AIRT,  described  in  the  story.  This  research  serves 
as  the  basis  for  attaining  the  "operability  vision"  Major  Tooms  referenced  in  his  talk  with  Captain 
Parrish  and  Lieutenant  Fillmore.  It  is  a  vision  shared  by  Major  Tooms  and  the  Air  Force  Human 
Resources  Laboratory. 
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The  acquisition  community  in  the  Air  Force  would  indeed  benefit  from  a  tool  or  methodology 
that  allows  dry-running  of  designs  before  prototyping  and  production.  The  benefits  of  such  a 
tool  include  effectiveness  and  efficiency  brought  to  human-centered  systems.  These  properties 
in  turn  would  bring  lower  costs  and  fewer  retrofits. 

But  what  should  this  tool  consist  of  to  satisfy  the  vision?  Several  components  are  required: 

-  A  rapid-prototyping  capability 

-  A  suite  of  configurable  operator-oriented  objects  that  include  visual,  auditory,  cognitive, 
and  psychomotor  models 

-  Scenario  development  tools 

-  Data  reduction  features  available  during  simulation  execution 

-  Post-processing  data  reduction  features 

To  complete  the  vision,  it  is  necessary  to  combine  the  components  above  in  a  manner  that 
allows  flexible  testing  of  virtually  any  human/machine  system.  Once  appropriately  combined, 
the  resulting  operability  testbed  and  assessment  tool  would  then  allow  rapid-prototyping  of 
systems  before  the  design  is  fixed.  These  quick  prototypes  allow  designers  to  create  a  "living" 
specification.  Such  specifications  shape  experiments  and  test  configurations  that  provide  valuable 
system  and  human  perfr.mance  data  to  the  designer.  In  turn,  by  using  these  data,  as  well 
as  expert  consultation  from  operators,  the  designer  can  respond  to  the  needs  of  the  operator 
(i.e.,  the  user)  and  adjust  the  specifications  as  required. 

These  needs,  while  focused  on  the  operator,  help  shape  and  develop  a  "system  maturity" 
(i.e.,  operability)  as  they  are  applied  in  various  scenarios.  Scenarios  exercise  the  test 
configurations  and  assist  the  designer  in  reaffirming  the  design  or  fostering  changes.  Throughout 
the  process,  the  data  reduction  features  offer  a  quantifiable  means  to  assess  the  impact  of 
automation,  or  of  lack  of  automation.  Furthermore,  the  operator  remains  a  key  player  in 
assisting  the  process. 

Once  the  appropriate  mixture  of  man  and  machine  is  agreed  upon  by  operator  and  designer, 
then  the  specification  becomes  more  binding.  However,  at  this  stage,  the  system  has  already 
been  through  extensive  test  and  evaluation  before  any  equipment  has  been  built.  Hopefully, 
by  using  the  operability  approach,  which  depends  heavily  on  modularity,  any  post-production 
changes  are  avoided.  If  not,  the  modularity  promotes  easier  modifications  without  the  need  to 
completely  redesign  the  system. 


EPILOGUE 

Again,  as  Major  Tooms  indicated,  the  payoff  is  a  better  system  that  is  more  responsive. 
Moreover,  the  system  is  longer  lasting  in  the  sense  that  it  does  not  have  to  be  replaced  quickly 
to  meet  new  threats  or  be  compatible  with  forthcoming  systems.  This  quality  promotes  less 
expense  over  the  course  of  a  system’s  acquisition  and  operational  life.  But  perhaps  most 
importantly,  the  operability  approach  and  assessment  of  automation  impacts  should  prevent  the 
system  from  being  unprepared  and,  thus,  risking  "embarrassment"  from  those  "operational" 
questions. 
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